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ABSTRACT 



The present invention relates to a method and apparatus for 
compressing digital data, such as the digital representation 
of a colored image. A scanning routine is described which 
aims to maintain the integrity of regions of the image that the 
human visual system also finds important. Pixel data from 
the image data includes color components, e.g., intensities of 
three or more colors such as red, green and blue or cyan, 
magenta and yellow. When the color components of the 
pixels are represented in color space, the axes of which are 
the three color components with the intensities thereof as 
variables, it has been found experimentally, particularly for 
artificial or synthetic images, that significant regions in the 
image tend to produce clusters of points in color space. In 
accordance with the invention, the color space is scanned by 
a space filling curve to convert the coordinates in color space 
into a one dimensional matrix such as a histogram. If certain 
types of space filling curves are used, e.g., a step-wise curve 
such as the Peano or Hilbert fractal, the clusters of points in 
color space tend to produce clusters of points in the histo- 
gram. Having obtained the histogram, it is segmented into a 
series of bins. These bins are used to segment the original 
image into regions which are coded. 

Although the invention has been described with respect to 
color images and 3-D color space, the method and apparatus 
may be useful with other types of data and with multidi- 
mensional spaces. 

21 Claims, 14 Drawing Sheets 
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DATA COMPRESSION METHOD AND 
APPARATUS 

RELATED APPLICATIONS 

This application claims priority from U.S. Provisional 
Application Ser. No. 60/000,965 filed on Jul. 7, 1995. 

BACKGROUND OF THE INVENTION 

1. Field of the Invention 

The present invention relates to image processing and, 
more particularly, to a method of compressing data, particu- 
larly visual data, and an apparatus for carrying out the 
method. 

2. Description of the Related Technology 

Today, digital information, both stored and transmitted, is 
pervasive. This digital information may be stored in the form 
of files, each file containing related data. When such a file 
includes visual data, such as a picture or a video, it may be 
tens or hundreds of megabytes, or even gigabytes in size. 

There is a general need to compress data for economic 
storage, analysis and transmission thereof without loosing 
the accuracy of the data or at least maintaining a functionally 
sufficient quantity of the data. The degree of compression 
necessary may vary. For video conferencing, a compression 
ratio of 100:1 or more may be required because of the 
limitations on data transmission by existing analogue or 
digital telephone lines. Similarly for the transmission and 
storage of large collections of still pictures via networks 
such as the Internet, very high compression ratios are 
advantageous. When visual data is stored as large files, there 
is a need to search in these files by a quick previewing 
method. Speed requires high compression and the final 
image need only be good enough to allow recognition of the 
subject matter. On the other hand for the compression of still 
picture of a natural scene, a 10:1 compression while main- 
taining very high fidelity could be a great advantage in 
reducing printing times. 

Many systems for compression have already been pro- 
posed. For picture data, two common schemes are Block 
Oriented Compression (B30C) and Region Orientated 
Compression (ROC). 

In BOC an image is processed on a block by block basis, 
where a block is normally a rectangular part of the original 
image. An example of BOC is the standard MPEG2 com- 
pression method for video and television. An individual 
block, which is a matrix of data in which each element of the 
matrix represents the color of a pixel of the image, is 
operated on by the Discrete Cosine Transform (DCT) to 
produce a further matrix defined by the coefficients of the 
DCT. The DCT coefEcients are quantized in order to reduce 
the number of bits required to code the coefficients and 
hence increase the compression. Further, the matrix with the 
quantized coefficients is sampled, usually in a diagonal 
zigzag fashion, and elements of the matrix which approach 
zero are ignored. At the receiving end, the image is recon- 
structed block by block using an inverse transform. The 
JPEG proposal for compression of still images also uses 
BOC and DCT. BOC using DCT is a preferred method when 
the degree of compression is low to medium (say less than 
80:1 for still color images) and natural images are to be 
stored or transmitted. The method becomes less successful at 
the extremes of compression and when artificial images are 
to be stored, displayed or transmitted. 

ROC involves the segmentation of an image into regions 
which are homogeneous with respect to a given image 
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characteristic; for example, segmentation into regions of 
uniform grey level or of uniform texture. ROC has the 
advantage that at least the theoretical possibility exists of 
segmenting the image into regions which may bear some 

5 relation to the parts of the image considered important by the 
human visual system (HVS). Theoretically, this could pro- 
vide an improvement over BOC which segregates the image 
into artificial rectangular blocks. Linked to ROC is often 
edge extraction of regions in the image. Segmentation into 
regions does not necessarily end up with the boundaries of 
the regions coinciding with edges, i.e., lines which define a 
significant intensity change. In conventional ROC schemes 
the region segmentation and edge extraction are carried out 
separately and each set of data is compressed separately. 
ROC has received wide coverage in the technical press 

15 but has not been successful commercially. One reason for 
this is that conventional ROC segmentation routines are 
computationally intensive. Only since the introduction of 
high speed computers has it been possible to consider ROC 
techniques. There still remains a requirement for a ROC 

20 method and apparatus which has high compression, is easy 
to implement, requiring electronics which can be imple- 
mented as VLSPs or PC-based software and which provides 
a final image which is adapted to the requirements of the 
HVS. 

25 The article by R. J. Stevens, A. F. Lehar and F. H. Preston 
entitled, "Manipulation and Presentation of Multidimen- 
sional Image Data Using the Peano Scan," IEEE Transac- 
tions on Pattern Analysis and Machine Intelligence, vol. 
PAMI-5, No. 5, Sept. 1983 describes a method of scanning 

30 color space using the Peano fractal curve. The method only 
relates to ordering and re-coloring data in a colored image. 
The amount of compression is minimal. The technique 
involves segmenting the histogram produced by the scan 
into a fixed number of bins and then re-coloring the pixels 

35 in the original image. The method is only suitable for 
pre-processing the image before conventional compression 
routines are carried out. 

SUMMARY OF THE INVENTION 

40 The present invention provides a method of compressing 
data comprising the steps of a) inputting a first matrix of 
digital data, each element of the first matrix being associated 
with a set of a plurality of "S" intensity values corresponding 
to "S" quantized levels of a characteristic of each element; 

45 b) determining an occurrence matrix on an element-by- 
element basis for the elements in the first matrix, each set of 
intensity values associated with an element in the first matrix 
representing a coordinate of a point in an "S" dimensional 
space, the "S" dimensional space having axes representing 

50 the intensity of the "S" quantized levels of the characteristic 
as variables, the position of an element in said occurrence 
matrix being determined by the distance from the origin to 
the coordinate of the point as determined by a step-wise 
space filling curve passing uniquely through each coordinate 

55 of the S-dimensional space; c) replacing each element of the 
first matrix by the distance from the origin in the 
S-dimensional space to the coordinate in the S-dimensional 
space representing the set of intensity values associated with 
that element to form a second matrix; d) segmenting the 

60 occurrence matrix into a variable finite number "P" of 
occurrence segments; and e) determining contour elements 
of first regions in said second matrix, said first regions being 
determined by segmenting the second matrix into first 
regions, the distance values in the S-dimensional space of 

65 each element of the second matrix in one first region of the 
second matrix falling into one of the "P 1 occurrence seg- 
ments. 
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This method also includes the steps of f) storing the passing uniquely through each coordinate of the 
contour elements as a third matrix, said contour elements S-dimensional space; a first frame buffer for storing the 
defining second regions in said third matrix corresponding to occurrence matrix; a segmenter capable of segmenting the 
first regions in said second matrix; and g) coding the occurrence matrix into a variable finite number "P" 
elements of one second region of the third matrix with a 5 of-occurrence segments; a second frame buffer for storing a 
representative set of values for the elements of said one second matrix, the second matrix being formed by replacing 
second region, each representative value being a distance each clement of the first matrix by the distance from the 
from the origin in the S-dimensional space to a coordinate in origin in the S-dimensional space to the coordinate in the 
the S-dimensional space, the set of representative values S-dimensional space representing the set of intensity values 
being calculated from the distances from the origin in the 10 ass0 ciated with that element; and a third frame buffer for 
S-dimensionai space to the coordinates in the S-dimensional storing contour elements defining first regions in a third 
space associated with the elements in the first region of said matrix, said first regions in said third matrix being deter- 
second matrix corresponding to the one second region. mined by segmenting the second matrix into second regions, 

The present invention also provides a method of decom- the distance values in the S-dimensional space of each 

pressing digital data, comprising the steps of step a) receiv- 15 element in one second region of the second matrix falling 

ing digital data representing the contour elements of regions mt o one of the "P" occurrence segments. 

in a first matrix and first characteristic data relating to a The apparams accon^g to the invention may also include 

characteristic of elements in each region of said first matrix; a coder for coding me element s of one first region of the 

step b) decoding said digital data to form said first matrix; mird matrix ^ a represe ntative set of values for the 

and step c) replacing the first characteristic data of each 20 elements of that first region, the representative value being 

element m each region of the first matrix by a set of a a & simcc from the orig in in the S-dimensional space to a 

plurality "S" of intensity values corresponding to "S" quan- coordinate in the S-dimensional space, the set of represen- 

tized levels of a second characteristic to form a second totive vahies being caiculated from tbe distances from the 

matrix, each first characteristic data associated with an orfgin m the S-dimensional space to the coordinates in the 

element in the first matrix representing the distance from the 25 s-dimensional space associated with the elements in the 

origin to a coordinate of a pomt m an "S" dimensional space secoad ion of the matrfx corr e S ponding to the said 

as determined by a step-wise space filling curve passing OQe first - on in ^ third matrix 

uniquely through each coordinate of the S-dimensional ™ . , . , . r , 

A. «™ ?■ i i_ ■ The invention also provides an apparatus for decompress- 

space, the S dimensional space having axes representing , . . . - . , , , „ 

J . \ . A - iL « 0 „ « j i i c Z j me data, comprising means for inputting digital data repre- 

the intensity of the "S quantized levels of the second 30 , t , ^ . . «„fL ♦ * 

. A . J . , j , • ,. A - 4i _ • A sentmg contour elements of regions in a first matrix and first 

characteristic as variabes and the coordinates of the pomt , * * j * i»- ♦ u * * * i 

^ iU , t(0 „ r . . , j characteristic data relating to a characteristic of elements in 

representing the plurality S of intensity values correspond- , . * -j « * . • c a a- -a 

- , «™ • j t ! c t . j » * • each region of said first matrix; means for decodmg said 

ing to the S quantized levels of the second characteristic. ^ , 0 form ^ flrs , matrix . ^ a oonw * r for 

The present invention also provides a method of com- sa id first matrix into a second matrix, said second 
pressing data comprising the steps of step a) inputting a first 35 malrix ^ fonned b rep i acing thc ^ characteristic data 
matrix of digital data, each element of the first matrix being of each el6mcnt m Mch re ^ on of ^ ^ matrix b , x{ of 
associated with a set of a plurality of 'S intensity values a luralit of « s „ mtensit values corresponding t0 « S » 
corresponding to S quantized levels of a characteristic of ^ levels of a xconA characteristic, each first char- 
each element; step b) determining an occurrence matrix by acterjstic data t8KjeilM ^ ln element in the ^ matrix 
scannmg the S imensKmal space having axes represent- 40 representmg tbe distanC e from the origin to a coordinate of 
ing the intensity of the S quantized levels of the charac- a • , m m „ s „ dimensional space ^ determined by a 
tensto as variables, each set of intensity values associated st ^ ^ ace filling curve passing uniquely through each 
with an element in the first matnx representing the coordi- co^^te o£ me S-dimensional space, the "S" dimensional 
nates of a point m the "S dimensional space; step c) havin axes represcnting me of me « s » 
segmenting the occurrence matrix into a variable finite 45 quantized levels of the second characteristic as variables and 
number "P of occurrence segmente; step d) segmenting the ^ coordinates of the point representing the plurality of "S" 
first matrix into regions, the set of intensity values of each mtensi yahles corresponding to me « S » quantized levels of 
element in one region of the first matnx representmg a ^ characteristiCj iai a frame buffer for ^ 
coordinate of the S dimensional space which falls into one second matrix 

of the "F* occurrence segments; and step e) assigning a 50 r~, • *j .1 j c 1 
representative set of intensity values to the elements in each ^ mention provides the advantage of a near-real time 
region of the first matrix, the representative set of intensity compression and decompression of large size images, 
values for each region being determined from the sets of Tte invention also provides the advantage of high corn- 
intensity values of the elements in that region in the first P ressioa ratios for or synthetic images. 
ma trix. 55 The invention also provides a method and apparatus for 
The present invention provides an apparatus for com- compressing data which may be easily implemented in VLSI 
pressing data, comprising a receiver for inputting digital technology. 

data in the form of a first matrix; a converter for determining The invention, its advantages and embodiments will be 

an occurrence matrix on an element-by-element basis for the described with reference to the attached drawings. The 

elements in the first matrix, each set of intensity values 60 dependent claims define further embodiments of the inven- 

associated with an element in the first matrix representing a tion. 

coordinate of a point in an U S" dimensional space, the "S" hdtct? nncnDiDTrnM r\u tut: no awtkt^c 

, 1 nvat , »ul ,- n , onr ;,„ n f BRIEF DESCRIPTION OF THE DRAWINGS 

dimensional space having axes representmg the mtensi ty 01 

the "S" quantized levels of the characteristic as variables, the The file of this patent contains at least one drawing 

position of an element in said occurrence matrix being 65 executed in color. Copies of this patent with color drawing 

determined by the distance from the origin to the coordinate (s) will be provided by the Patent and Trademark Office 

of the point as determined by a step- wise space filling curve upon request and payment of the necessary fee. 
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FIG. 1A shows the transform of an arbitrary image (here 
shown as a photograph of a person's head when lying down) 
into color space in accordance with the present invention. 

FIG. IB shows a histogram produced in accordance with 
the method of the present invention. 

FIG. 2 is a block diagram of a grey scale compression 
routine in accordance with the present invention. 

FIG. 3 is a block diagram of colored image ROC routine 
in accordance with the present invention. 

FIG. 4 shows a Hilbert fractal curve. 

FIG. 5 shows a first embodiment of the ROC scheme in 
accordance with the present invention. 

FIG. 6 shows a histogram segmented into bins in accor- 
dance with the present invention. 

FIG. 7 shows a segmented image (shown here to corre- 
spond with the head image of FIG. 1A) in accordance with 
the present invention. 

FIGS. 8 and 9 show schematic diagrams of finite state 
machines as used in accordance with the present invention. 

FIG. 10 is a block diagram of an ROC scheme in 
accordance with a second embodiment of the present inven- 
tion. 

FIG. 11 is a block diagram of a computer system in 
accordance with the present invention. 

FIG. 12 to 17 show flow diagrams of the method of the 
second embodiment of the present invention. 

DESCRIPTION OF THE ILLUSTRATIVE 
EMBODIMENTS OF THE INVENTION 

The following detailed description of the preferred 
embodiments presents a description of certain specific 
embodiments to assist in understanding the claims. 
However, the present invention can be embodied in a 
multitude of different ways as defined and covered by the 
claims. Reference is now made to the drawings wherein like 
numerals refer to like parts throughout. 

In the following text, the present invention will be 
described with reference to compression of graphic images 
and pictures. The invention is not limited thereto. The 
invention may be used with any data which can be repre- 
sented as digital data, whereby a characteristic of each 
element of the digital data may be represented by a set of 
intensity values, e.g., not only color data but also audio data 
may be represented as a set of intensity values of quantized 
frequency ranges. The invention is also described generally 
with reference to a three component representation of the 
color of each element of the digital data. The invention is not 
limited thereto. The present invention is suitable for use with 
a two component as well as with a four or more component 
representations of color. For instance, the invention is suit- 
able for use with the additive three component RGB (red, 
green, blue) system of color representation or generation as 
well as with the subtractive CMY (cyan, magenta, yellow) ss 
or the four component CMYK (cyan, magenta, yellow, 
black) system of color representation or generation. The 
invention is suitable for use independently of the number of 
intensity values required to represent the characteristic of 
each element of data. Further, the invention will be 
described with reference to a Cartesian color space. The 
invention is not limited thereto. For instance, the invention 
may be used, with the necessary modifications to the space 
filling curve, with a polar or cylindrical color space, e.g., the 
coordinate system developed by the CIE (Commission Illu- 
mination d'Eclaraige) in 1976, for further details see A. L. 
Robertson, "The CIE 1976 color difference formulae" , 



Color Research and Application, vol. 2, no. 1, 1977, pages 
7-11 or K. McLaren, "The Development of the CIE (L*, a*, 
b*)— uniform color space" , JSDC, Sep. 1976, pages 338 to 
341 or the empirical Munsell color order system, for further 
details see G. Wyszecki and W: S. Stiles, Color Science, 
New York: Wiley, 1967. 

Any color in a color image recorded on an analogue 
medium, e.g., a photograph or a dispositive (a positive 
formed on a transparent support such as glass), may be any 
of an unlimited number of colors. Each color is electromag- 
netic radiation defined by the scalar quantity of its fre- 
quency. If such an image is digitized it is represented by a 
finite number of individual elements or pixels A which may 
be represented by an MxN image matrix: 
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It is well known that it is possible to represent a color by 
the sum of at least three primary colors, alternatively the 
subtraction of three primary colors from white light. Even 
combinations of two colors may give a colored image which 
does not contain true colors but may be sufficient to give an 
approximately realistic impression. The number of colors 
which may be represented is usually quantized, e.g., with an 
8-bit representation of a grey scale, 256 different shades of 
grey may be represented whereas with a 24-bit color 
representation, in which each component of the RGB or 
CMY has 8 bits, theoretically, 16.8 million discrete colors 
may be represented. Each pixel may be defined by a set of 
intensity values of the primary colors, e.g., intensities of red, 
blue, green (RGB); cyan, magenta, yellow (CMY); or cyan, 
magenta, yellow and black (CMYK). The primary colors are 
referred to in this application as the quantization levels or 
"components" of the characteristic of the elements of the 
image matrix. Generally, each pixel may be represented by 
a set S of intensity values I, corresponding to S quantization 
levels or components of the particular characteristic "color". 
Hie complete image is then representable by S, MxN 
matrices: 



45 



50 



= matrix 1 



=s matrices 2 to S - 1 



k 'am 



60 where is the intensity of the quantization level or 
component (i.e., one of the primary colors) of the charac- 
teristic property (here the color) of the pixel or element Axy 
of the image matrix. For example, for an RGB representa- 
tion of the color of each pixel, S equals 3; for the CMYK 
representation, S equals 4. 

The color of each pixel A^ may be represented as a 



coordinate (I x , 



, I s ) in a color space whose axes are the 
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intensities of the S components of the characteristic as a small latency, mainly to limit the required intermediate 

variables. Pixel data from images will normally tend to form storage and input buffering, especially if the image segmen- 

clusters in color space, each cluster relating to closely tation is part of a feedback loop (e.g., when iterating the 

related colors of one object in the image, e.g., flesh tones in segmentation towards a specified number of contour pixels, 

the face of a person, the blue tones of the sky on a sunny day. 5 In accordance with the present invention a single 

FIG. 1A shows a three dimensional color space (RGB or segmentation-single coding scheme is preferred. These 

CMY, for instance) in which pixel color data of an arbitrary schemes combine the components (preferably in a reversible 

image is shown. The clusters 1, 2, 3 relate to three distinct way) into one component, used for both segmentation into 

objects in the original image. The original color image in this regions and segment coding to produce the coded contour 

example is of a person's shoulders and head when lying 10 P«el data. FIG. 3 is a schematic diagram of a single 

(j own segmentation-single coding scheme in accordance with the 

In the following detailed description of the invention, present invention. In the coder, the color separated RGB data 

non-limiting reference will be made to three component H, 12, 13 from the input image (generally, the intensity 

color representation systems, e.g., RGB or CMY, as it is values of the S components of each element of data) is 

easier to describe, display and understand three dimensional is represented in 3-D color space (generally, S-dimensional 

spaces. Where appropriate, reference will be made to the space) and transformed in a 3-D to 1-D function 14 into a 

more general S dimensional space of an arbitrary charac- 1-D matrix 15 by means of a suitable algorithm. Clusters of 

teristic of the data represented by the intensities of S data in the 1-D matrix 15 are used to define segments of the 

components. matrix called bins. The dividing lines between bins in 

In ROC, an attempt is made to segment the image matrix 20 the 1-D matrix 15 are used for the segmentation function 16 

into regions which bear a close relation to the objects in the to produce a segmented image 17. The segmented image 17 

image perceived by the human visual system (HVS). ROC has a finite number of regions. Each region in the segmented 

basically consists of a segmentation stage followed by a image 17 is defined by contour pixels. Contour pixel data 

coding of the regions. FIG. 2 shows a flow diagram for grey from the image 17 is output to a communication channel 20. 

scale or one-component images in accordance with the 25 The 1-D representation 15 is used with the region data from 

present invention. The input image 5 is segmented in a the segmented image 17 to determine the color associated 

segmentation function 6 into regions of uniform levels of with each region in a segment coding function 18 based on 

grey to form a segmented image 7. Grey scale data relating the data in that region. From the segment coding function 18, 

to each of the regions from image 7 is determined in a the region color data is output as a 1-D bit stream 19 to the 

segment coding function 8 and is output via coded segments 30 communication channel 20. On receipt of the data, the 

9 to the communication channel 10. The contours of the decoder (not shown) performs the reverse procedure, 

regions are coded to form contour output data which is The compression ratio CR of such a scheme may be 

outputted to the communication channel 10. expressed as: 

Advantageously, ROC may produce high compression 

ratios, particularly in artificial or computer generated 35 CR _ 3xf>xNxAf 

images. At low compression ratios, BOC may be preferred ~ B c xc + 3xbxr 
because of its low computational and implementational 

complexity. High compression implies that locally, at the wnere - 
edges of the regions, large errors may occur. Since the HVS 

masks errors that coincide with object borders, it is advan- 40 b ^presents the number of bits used to represent a color 

tageous if the segmentation stage produces regions with component, 

edges coinciding with the borders of the objects recognized N > M represents the image dimensions, 

by the HVS. This is particularly important in the compres- B c represents the number of bits to represent a contour 

sion of 3-D images. 3-D still, television or video images are pixel, 

usually produced from two normal images which are viewed 45 c represents the number of contour pixels, 

separately by each eye. This may be achieved by each r represents the number of regions, 

stereoscopic image being orthogonally polarized with The contribution of the coding of the contour pixels in 

respect to the other and the viewer wears glasses having highly compressed images should not be neglected. Differ- 

polaroid lenses whereby the axes of polarization of the two ent coding rates of contour pixels can be found, e.g., 3 

lenses are orthogonal to each other, or by green and red 50 bit/contour pixel, H. Freeman, "On the encoding of arbitrary 

images and the viewer wears glasses with green and red geometric configurations" , IRE Trans. Electron. Comput., 

lenses. Alternatively, the two stereoscopic images may be vol. EC-10, pages 260-268, Jun. 1961, down to 1.27 bit/ 

formed by alternate lines of an interlaced television picture contour pixel, M. Eden, M. Kocher, "On the performance of 

and the viewer has glasses which have light valves opening a contour coding algorithm in the context of image coding:, 

and closing synchronously with the lines. The HVS relies 55 Parti: Contour segment coding" , Signal Process., vol. 8, 

upon edge discrimination of objects in order to generate the pages 381 to 386, 1985, all of which documents are incor- 

3-D effect. If the segmentation results in relative movement porated herein by reference. Experience has shown that if all 

of edges of an object in the two images, the position of the overhead (e.g., pointers to different regions) is included and 

object in the perceived 3-D image changes. This might severe constraints (no non-connecting contours, only 

result, for instance, in the disconcerting effect that one of the 60 smooth regions) are rejected, a realistic coding results in no 

eyes of a person might be behind or in front of the head. less than 1.9 bit/contour pixel (see also T. Kaneko, M. 

It is advantageous if the latency which is introduced by Okudaira, "Encoding of Arbitrary Curves Based on the 
the ROC system is kept to a minimum: latency is important Chain Code representation", IEEE Trans. Com. COM-33, 
for interactive or near real-time systems. Latency is the pages 697 to 706, 1985, incorporated here by reference), 
delay between inputting and outputting data and is a mea- 65 Consider the following typical values: b=*8,N=M=256,B c _ 
sure of the amount of data which must be stored during 1.9,c=4000/=250. The compression ratio of a single- 
processing. A desirable segmentation algorithm should have segmentation scheme is 116:1 with a relative contribution of 
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56:1 from the contour pixels. The conclusion remains valid preferred if the step-wise space filling curve is such that data 

that, when dealing with high compression ratios, the relative close together in the 1-D transform corresponds to pixels 

contribution of the contour pixels is high. A single- close together in the image matrix. The latter property is 

segmentation single-coding scheme is preferred in accor- important since high compression implies that the coding 

dance with the present invention. Both the address genera- 5 stage introduces errors, which could be amplified signifi- 

tion complexity and the number of memory accesses are cantly in the decoder by discontinuities in the 1-D to 3-D 

decreased in a significant way, since only one segment conversion. It is preferred if the stepwise space filling curve 

coding process exists. Hence, only one (instead of three) has certain properties of a either a deterministic or a random 

process accesses the segmentation information and the inter- fractal. In particular, it is advantageous if the space filling 

mediate memory needed for the segment coding. It has been 10 curve has deterministic or statistical self-similarity. For 

shown that memory accesses and address generation are the further details of self-similarity of space filling curves, 

bottle-necks when implementing complex image processing especially fractals, see "Fractals and image representation" 

algorithms. When realizing the scheme in VLSI, the inter- , R. J. Clarke and L. M. Linnett, IEE Electronics and 

connection overhead is reduced in the single-segmentation Communication, vol. 5, No. 4, Aug., 1995, which is incor- 

single-coding scheme thanks to the reduction of functional is porated herein by reference. Self -similarity may provide the 

building blocks. The 3-D to 1-D conversion in accordance advantage that if the number of regions of the image are 

with the present invention in the coder (and the 1-D to 3-D. varied to maintain the compression ratio within acceptable 

conversion in the decoder) introduces a small overhead. levels, the loss of some of the regions will leave the 

Therefore, a single-segmentation single-coding scheme is remaining regions in the same place (the regions are orga- 

performed. 20 nized in a hierarchical tree structure). This property allows 

Different one-dimensional segmentation spaces have been better control of the compression ratio by varying the 

proposed, such as the intensity and the hue. The transforms number of regions used for the segmentation. It is also 

to these spaces have the disadvantage that they are not preferred that the length of the step-wise space filling curve 

reversible, so that the coding stage can not be performed in from the origin to any arbitrary coordinate in the color space 

these spaces (e.g., these segmentation is performed on a 25 may be calculated by simple arithmetical routines. This 

gray-scale version of the image, and the coding is computed simplifies the transformation of pixel data in the image 

according to this segmentation in the color space). Multi- matrix to the 1-D occurrence matrix, 

dimensional space to one-dimensional space transforms For example, in accordance with the present invention, 

such as performed by function 14 of FIG. 3 in accordance the 3-D to 1-D transform may be calculated by scanning the 

with the present invention are preferably reversible. In 30 3-D color space with a Hilbert fractal (often called the Peano 

accordance with the present invention the multi-dimensional curve) scan or a dovetail fractal scan or a similar scan based 

color space is sampled by a step-wise space filling curve on a step- wise form filling curve as defined above, 

with unique mapping to the 1-D space to generate an Alternatively, the transform may be made by a raster scan, 

occurrence matrix (generally, the S-dimensional space is It is thus a priori possible to perform the segmentation and 

sampled by a step-wise space filling curve with unique 35 the segment coding in the same one-dimensional space. The 

mapping into the 1-D space). The simplest form of occur- raster scan and the Hilbert scan are described in "Data 

rence matrix is a histogram. For each pixel A^ of the image ordering and compression of multispectral images using the 

matrix, the distance L from the origin in color space to the Peano scan" , R. J. Stevens, A. F. Lehar and F. H. Peterson, 

coordinate (I 1 I^y 2 , L^ 3 ) of this pixel's color along the IEE International Conference on Electronic Image 

step-wise space filling curve is calculated. 40 Processing, pages 209 to 213, Jul., 1982, see also "Amethod 

The distance L defines the X axis in the histogram shown of pre-processing color images using a Peano Curve on a 

in FIG. IB as a variable. The Y axis is defined by the number Transputer array," R. Lambert and B. Batchelor, SPIE Vol. 

of pixels in the image data which have a color which has a 1381 Intelligent Robots and Computer vision IX: Algo- 

certain length L when transformed into the 1-D space. The rithms and Techniques, pages 582 to 588, 1990 and 

histogram defines the number of times that a particular color 45 "Manipulation and presentation of multi-dimensional Image 

occurs in the image data and is therefore an occurrence data using the Peano scan" , R. J. Stevens, A. F. Lehar, and 

matrix (0 L1 , . . . ,0 LW ), where 0 Lj is the number of color F. H. Preston, IEEE Transactions on Pattern Analysis and 

occurrences at the distance L y from the origin of the color Machine Intelligence, vol. PAMI-5, No. 5, Sept. 1983, and 

space along the step-wise form filling curve. The invention the dovetail fractal is described in "Mapping Multidimen- 

is not limited to simple histograms. Occurrence matrix (O), 50 sional Space to a one Dimension for Computer Output 

may be multi-dimensional. For example, the occurrence may Display", IEEE Transactions on Computers, vol, C-17, No. 

include data relating to weighting factors for particular 10, pages 949 to 953, Oct. 1968, all of which are incorpo- 

lengths L. For instance, if cartoons or maps are to be coded, rated herein by reference. 

the lengths L corresponding to the colors of certain lines, The occurrence matrix calculated by means of the raster 

e.g., black lines in a cartoon or the colors of roads on a map, 55 scan results in each component of the color representation 

may be given a different weighting so that these are pref- being distributed along the length L rajrer in a line. The 

erentially selected as regions in the image. occurrence matrix calculated by means of the Hilbert scan 

In accordance with the present invention, it is preferred if results in the color of each pixel being represented as a 

the step-wise space filling curve is an unbroken curve which length L Hilbert traversed by the Hilbert fractal curve from the 

passes through every element in the color space. It is 60 origin to the coordinate of the color in color space. As an 

preferred if each step of the curve ends and begins on example R-2, G-0, and B-3, as shown by reference number 

coordinates in the color space which coincide with the 22 results in an L HUben of 59 units as shown by the dashed 

coordinates of the quantized color representation, but the line in FIG. 4. 

invention is not limited thereto. It is further preferred if the The impact of the errors introduced in the coding stage 

step-wise space filling curve is such that pixels close 65 has been calculated by modelling these errors in a first 

together in the image matrix correspond to data in the 1-D approximation as a loss of significance of the least signifi- 

transform which is close together or clustered. It is also cant bits of the distance L. The entwinement of the bits in a 
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Hilbert scan results in a uniform spreading of the compres- shown in FIG. 6. If the number of groups or bins 39 thus 

sion errors over the color components, whereas using the formed is less than or equal to a predetermined value, these 

raster scan concentrates the errors into one color component. bins 39 are accepted. If there are too many bins 39, the less 

The robustness of the Hilbert scan with respect to compres- important bins 39 are merged. For instance, the bin 39 (in 

sion errors is a result of its self-similar or multi-resolutional 5 FIG. 6 this would be the bin between dashed boundaries 2 

fractal nature: the loss of least significant bits corresponds to and 4) with the smallest number of entries in the bin is 

a coarser detail level of the color space. merged to its neighbor thus reducing the number of bins 39. 

FIG. 5 shows a Hilbert fractal-based ROC scheme 30 as This procedure is repeated until the required number has 

a preferred embodiment of the present invention. The digi- been reached. Other criteria for merging may be used as are 

tized input image 25 is analyzed pixel by pixel. The color 1Q known to the skilled person. 

coordinates 26-28 of each pixel are converted into a The segmentation scheme performed by the segmentation 

length ^naber?* m col 01 * sP acc bv a converter function 29 unit 33 (FIG. 5) in accordance with the present invention 

and output in a matrix 31 as a length L^^J^on a pixel by requires at least two memories: the LHilbwrt input frame 

pixel basis. Each length h„ abmr ^ Y corresponding to a pixel buffer 34 (FIG. 5) and the histogram memory of the histo- 

Ayy is fed to a histogram manager 32, which keeps the g ram manager 32 in the histogram manager 32. The l* Hiiben 

histogram of the current image up to date and stores it, and 15 kp ut frame buffer 34 has the same dimensions as the image 

to the frame buffer 34 in which the image is stored as a dimensions (NxM), and each pixel requires three times the 

matrix, with each pixel defined not by its color but by its ™°** r of used £ "^put color components (typically 

^Hubert value. Peano scan histograms are described in 3x8 j * uscd m reversible apphcaUons, e.g., ROC, or less if 

«w • 1 *■ j * » %u t xy « • « used in non- reversible applications, e.g., preprocessing for 

-Manipulation and presentation. Tie matnx 31 2Q recognition. The number of bins 39 in the histogram 

can be computed by the function 29 w an .teraUve way, three de ^ nds ^ Ucitioa ^ k related to ^ maximum 

bits at a time as described in Space Ming ; curves: Their re £ lution ^ has tQ achieye „ can be k 1qw 

Generation and Then- Application to Bandwdth reduction , manks (0 ^ dvsictia rties of the Hilbert fractal 

IEEE Transactions 0° formation Theory, vol U-15, pages aaDs{ resultin in memory requirements for the 

658 to 665 Nov., 1969 which is incorporated 1 herein by ^ hologram. Experiments have shown that the number of bins 

reference. One iteration step operates on one bit of each - n _T • , c n r\r^ • * „• n 

i * ..t. i .* 1 i, a.., %• x 39 required for ROC is typically 

color component at the same resolution level k (bit position): ^ 

k=0 corresponds to the coarsest resolution level, k=b-l 
corresponds to the highest resolution level. The generating 

formulae are: 3Q 



Vc x Vc- 



7t = G[?k * Tk\ where C is a constant which may be decided arbitrarily 

= = = depending upon the application and may be 1,2,4, etc. and 

* 55 new{ k ~ l) 1-1 is typically 4. The word length (^number of bits) required 

35 for each bin equals : logs (NxM). When using 256x256 

with: images, the histogram memory in the histogram manager 32 

X* representing the input vector for the resolution level k between 1 and 4 Kbits For ROC an additional 1 

T representing the transform matrix j* * xM frame buffer is needed to store the L segmented 

G representing the bit mapping transform 4Q ^'bins 39 in the histogram shown in FIG. 6 are divided 

Y representing the output vector for the resolution level k m to G groups that are separated by G-l edges. In FIG. 6 the 

Tk * X is defined as follows: bins are snown as a continuous histogram distribution, the 

boundaries between the bins are labelled 1 to 4. Using the 

yi . x z 7*Y boundaries 1 to 4 between the bins 39, the data in the frame 

p£ ,J li% * 45 buffer 34 is converted into the segmented image and stored 

in the frame buffer 35 using the segmenting unit 33. 

'o = 1 -> Uj Oxj=xj A segmented image in accordance with the present inven- 

r i; = 0-*r l7 ox; =0 tion is shown in FIG. 7. The exemplary image 35 corre- 

t t =i -l z r ojcj ; = l - xj sponds to the original image of a person's head when lying 

u 'j j j 5Q down p rev i ous iy shown in FIG. 1A. The segmented image 

contains only the contour pixels of the regions, e.g., the 

X* is a three bit vector consisting of one bit of each contours 42 are black and the regions 43 are white. The 

color component at the resolution k. The iterative contours are defined by adjacent pixels in the image which 

generation process is started at the coarsest detail lie in two different bins 39 of the histogram. The segmented 

level (most significant bit of the color components), 55 image in accordance with the present invention shown in 

vjeldingY 0 ,the three most significant bits of L mberr . FIG. 7 was produced using 10 bins. The segmented image 

T 0 is the identity matrix, T nw is computed from Y 0 may be output to a communication channel 40 via a contour 

so that T j is known. The three bits of the next pixel coder 38. This coder may code the contour pixels in 

resolution level are processed until the required accordance with any conventional technique, for instance as 

resolution is reached. 60 described in U.S. Pat. No. 5,459, 513, or "Region Image 

Analogous formulas are used for the inverse transform Coding," D. Moran, T. Vlaches, and A G. Constantinides, 

used in decoding. presented at the IEE colloquium on Low Bit Rate Image 

Once the histogram is computed, an adaptive histogram Coding, London, May 1990 or similar methods already 

multi-threshold technique may be used to divide the histo- mentioned above. 

gram into segments or bins 39 as shown in FIG. 6. As an 65 The segmented image stored in frame buffer 35 may be 1 ; 

example of such a technique, the minima in the histogram sufficient for pattern recognition purposes. However, fbr I 

are determined and the boundaries set at these points as reproduction of a color image, the data from segmented J 
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image is supplied to a region coder 36 configured as an The segmentation function of FIG. 5 alone introduces no 
arithmetical unit. This coder 36 takes the data from the frame latency, uses no local memory, and thus does not require 
buffer 34 for each region and calculates the appropriate memory accesses, nor address generation. The price which 
representative color to assign to this region. T he represen- has to be paid is the use of the histogram manager (FIG. 5) 
tative color for a region may be determined by a statistical 5 and 3-D to 1-D converters 29. The latency introduced by the 
manipulati on^)^ elements of frame buffer complete ROC scheme is two frames. The required storage 
J4 in trie particuIarregioTTFor, insta nce an average value or equals one 24-bit frame 34 for the 1-D image, one 1-bit 
a median valuemay be ' calculate^. A lternatively ~an3~ frame 35 for the segmented image, and the histogram 
f ^f^riblyTan*an3ys1^ to determine the memory in the histogram manager 32. In these consider- 
most commonly occurring color in the region. Alternatively, 10 ations the latency introduced by the region coding and the 
the representative color may be calculated by minimizing storage required by the region coder have been ignored, 
the root mean square of the differences between the value of since they depend on the selected coding technique, 
each element in the region and the representative value or The ROC scheme in accordance with the present inven- 
any other appropriate averaging technique may be applied. tion may be implemented in software or may be hardware, 
In accordance with the present invention the determina- 15 e.g., in a VLSI -Implementation. Referring to FIG. 8, the 
tion of the color to be assigned to each region is determined I-HUbert generating formulae can be modelled as a Finite 
from the color data in that region rather than assigning an State Machine (FSM) 50 with one bit of each color corn- 
average color value to a bin 39 (FIG. 6). In accordance with ponent X* at resolution level k as input 51, resulting in 
the present invention the colors of regions belonging to the output 52, Y A having three successive output bits and an 
same bin 39 may have differing colors. Thus, for instance, a 20 internal state 53. The FSM can be considered a hardware 
particular bin 39 may have an average color brown with implementation of the 3-D to 1-D Hilbert converter 29 (FIG. 
extremes of dark brown and light brown within the same bin. 5) and provides an iterative way of obtaining distance. 
A region whose contours have been determined by the Computation using this technique requires eight clock 
boundary between this bin and the next bin which is, say, cycles. The pixel rate is thus eight times less than the clock 
red, may be assigned any color between dark brown and 25 frequency of the circuit. 

light brown depending upon the dominant color in the Referring to FIG. 9, a higher pixel rate can be achieved by 

region. This has a profound effect on color fidelity. Where an chaining different FSM's 54, 55. The input 56 to the first 

image contains many small regions, each region in accor- FSM 54 is with an output 58 of Y^ a . In the second FSM 

dance with the present invention has a degree of indepen- 55, the input 57 is X fr and the output 59 is Y A . A bit level 

dence. For instance, in the example given above with r-250, 30 pipelining of this chain can be considered, which introduces 

the number of bins may be as low as 5 or 6. Thus on average latency but preserves the high pixel rate, 

there are about 50 regions for each bin. Each of these regions The Hilbert converter in accordance with the present 

has color independence within its bin. The final compressed invention has been implemented in 0.7 /«n double metal 

image has a color quality as if it has been segmented with CMOS technology. The design includes 1942 equivalent 

more regions than has actually been the case. 35 NAND gates, resulting in a silicon area of 1.7 mm 2 . For true 

Alternatively, sets of data may be used as the represen- color (24 bit) applications, pixel rates up to 27 MHz have 

tative values. This may allow some hue variation within a been obtained. 

region. For instance, using techniques such as Fast Fourier A second embodiment 60 of the present invention will be 

Transforms (FFT), DCT or special techniques as described described with reference to FIGS. 6 and 10. The bins 39 in 

in the thesis "Vervonnde Polynomen voor de Compressie 40 the histogram shown in FIG. 6 are divided into G groups that 

van Een- enTweedimensional Signalen", by Wilfred Philips, are separated by G-l edges. In FIG. 6 the bins are shown as 

University of Ghent, Belgium, 1992-1993, the color varia- a continuous histogram distribution, the boundaries between 

tions in the data in each region may be approximated by a the bins are labelled 1 to 4 in accordance with their intro- 

polynomial, whereby only the coefficients of the polynomial duction. Using the boundaries between the bins 39, the data 

need to be output. To reduce the number of bits required, the 45 resulting from the 3-D to 1-D conversion is converted into 

coefficients may be quantized and insignificant coefficients the segmented image stored in frame buffer 35 by the 

discarded. Alternatively, the variations of color within one segmenting unit 33. The contour pixels are those which lie 

region may be approximated by the conventional technique adjacent to a pixel in another bin 39. These G bins 39 

of color dithering. The data relating to the representative correspond to at least G regions in the segmented image. For 

color values or sets of values of the elements in the regions 50 each bin 39 there may be many regions. On increasing the 

is output to the communication channel 40 as a bit stream 37. number of bins G, the details of the regions will be refined. 

The decoder in accordance with the present invention per- If precaution is taken to only place new boundaries in the 

forms the reverse operation whereby the input data including histogram, without changing the location of previously 

. the coded contour pixel data is first decoded and stored in a placed boundaries, only new regions will appear in the 

frame buffer 35 and the region color data is used to create the 55 segmented image when increasing G. When G is decreased, 

^miben "nage* stored in a frame buffer 34 and transformed first regions coinciding with the finest details will be merged 

into a color image via a 1-D to 3-D transform. into the larger background region, 

A test image of a portrait coded with 7 bins in accordance The images resulting from a segmentation with G and 

with the present invention having a compression ratio of G-l bins 39 in the histogram have a common subset of 

130:1 was compared with the same image compressed by 60 regions. This means that a tree-like hierarchical segmenta- 

DCT and BOC having the compression ratio of 108:1. tion is performed, with every region being the root for the 

Whereas the portrait was recognizable with the compression next resolution level k. A gradual and consistent detail level 

scheme of the present invention, BOC and DCT resulted in increase or decrease is hence made possible, 

a meaningless picture at this compression level. The original In FIG. 10, the same reference numerals have been used 

image required a total of 192 Kbytes whereas the com- 65 for the same functional items as in FIG. 5. As described in 

pressed image in accordance with the present invention conjunction with FIG. 5, this second embodiment 60 

required 1508 bytes. includes the input image 25, the segmentation unit 33, the 
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segmented image frame buffer 35, the region coder 36, bit 
stream 37, and the communication channel 40. The first 
segmented image stored in a frame buffer 35 is produced 
using the method in accordance with the present invention 
from the input image 25 based on default values for the 5 
determination of the number G of bins 39. Segmentation 
statistics, e.g., number of regions, number of contour pixels, 
are extracted in a statistics function 62 and compared in a 
comparator 63 with the required or maximum/minimum 
compression level, which may be input from an input device to 
64. As the number of contours is largely dependent on the 
number of regions, this data may optionally be input to the 
comparator 63 from the region coder 36. If the compression 
ratio is too high, the parameters for the segmentation may be 
changed by a parameter circuit 61, e.g., reduction in the 15 
number of bins 39, and a new iteration triggered. The 
parameter circuit 61 uses feedback to implement region- 
oriented compression. Operation of circuit 61 is shown in a 
portion of the steps of FIG. 12. 

In accordance with the second embodiment, the compres- 20 
sion ratio can be maintained over a certain minimum. This 
is advantageous in the transmission of video or still images 
when the transmission bit rate is limited, e.g., for video 
conferences over conventional telephone lines. By adapting 
the segmentation it is possible to control the bit rate at the 25 
loss of some definition. 

A modification of the apparatus in accordance with the 
second embodiment may be used for person or pattern 
recognition. If the person may be recognized at a low level 
of resolution k, the recognition procedure is terminated 30 
immediately. If, however, there is doubt as to recognition, 
the resolution level may be increased manually by setting 
the number of bins through circuit 61 and the image 
becomes more detailed. Long times are, thus, only required 
when recognition is difficult. 35 

As a further modification, the invention may be imple- 
mented in software and used by object linking with com- 
puter based word processing systems. In such an implemen- 
tation the segmentation of the histogram could be done by 
the operator until a satisfying result is obtained. 40 

FIG. 11 illustrates a computer work station generally 
indicated at 70 which is representative of the type of 
computer that is used with the present invention. The 
workstation 70 comprises a computer 71, a hard disk drive 
72, a color monitor 73, a mouse 74, a keyboard 75, and a 45 
printer 76. In the presently preferred embodiment, the visual 
display unit 73 is a 15 inch color monitor having a 1024x768 
pixel resolution. The mouse or other pointing device 74 is 
used among others things to "pick" an image entity. The hard 
disk drive or other mass storage 72, in one possible con- 50 
figuration of the workstation 70 stores 1 gigabyte of data. 

The fundamental processing components of the computer 
71 are a microprocessor such as, for example, a Pentium 
processor manufactured by Intel and a main memory 
comprising, for example, 16 megabytes of random access 55 
memory (RAM). The RAM in preferably configured as a 
video ram, e.g., a dual port memory. The RAM is preferably 
used to temporarily store the segmented image data, the 
histogram, and the ID image of L Hilberr The main memory 
is used to store certain software including the operating 60 
system and drivers for the hardware as well as the encoding/ 
decoding software written in the C language and using a C 
compiler available from Microsoft Corporation. The 
encoding/decoding software includes the 3-D to 1-D 
converter, a histogram manager, the image segmentation 65 
routine and the region coding routine. Computer 71 may also 
include input/output devices 78 fbr communication with a 
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network or communication system 79 such as the Internet. 
The computer 71 and storage 72 are preferably located in an 
enclosure 77. 

Although a representative workstation has been shown 
and described, one skilled in the applicable technology will 
understand that many other computer and workstation con- 
figurations are available to support the present invention. 

FIGS. 12 to 17 show flow diagrams of the method in 
accordance with the second embodiment of the present 
invention. Referring to FIG. 12, the compression routine 
starts with step 100. Step 150 includes reading the param- 
eters and limits for the application, e.g., the maximum 
number of bits allowed for the final image. In step 200, the 
default value of bins (G) for the histogram is selected, e.g., 
the previous value is taken or a set value. Step 250 includes 
first encoding routine A, which includes steps 251 to 260. 

Referring to FIG. 13, with step 251 the image data is read 
in and prepared for processing, e.g., removing headers and 
the histogram is zeroed. In step 252 one of the pixels of 
image data is read in and converted in step 253 from the 
color space defined by the color components of the pixel into 
a distance L in color space determined by the Hilbert fractal 
scan. In step 254, the histogram is updated with the new 
value of ^ HiIberr In step 255, this new distance L miben is 
then added into the frame buffer (34 of FIG. 5) storing the 
representation of the image described by L Hilberr , values. In 
step 256 it is determined if the image has been completely 
scanned. If not, the steps 252 to 256 are repeated for the next 
pixel. Once the image data has been completely read in, the 
histogram is complete and so is the L HiIbert image in the 
frame buffer 34. In step 257, the histogram is segmented into 
bins in steps 701 to 710 shown in FIG. 15. 

Referring to FIG. 15, the histogram segmentation routine 
is started at step 701. In step 702, the minima in the profile 
of the histogram are determined. In step 703, the number of 
minima found, M, is compared with the maximum number 
of bins allowed, M ma:r . If M exceeds M max , the smallest 
histogram segment F is determined in step 705. In step 706, 
the two neighbors of segment F are compared, and the 
smallest segment Z of the two is determined. In step 707, 
segments Z and F are merged thus reducing the number of 
segments to M-l (new M). In step 708, the new M is 
compared with M maa .. and steps 705 to 708 repeated if the 
new M still exceeds M max . If the new M is less than or equal 
to M max , the boundaries determined by the minima are taken 
as the boundaries of the bins in the histogram at step 709. 
Hie routine returns at step 710 to step 258 of FIG. 13. At step 
258 (FIG. 13), the bins are used to segment the h Habert 
image stored in the frame buffer as shown in steps 801 to 
811, FIG. 16. 

Referring to FIG. 16, the routine is initialized at step 801. 
In step 802, a pixel of data is read from the L H£Iben image in 
the frame buffer 34. In step 803, the bin j of the histogram 
to which the particular value of ^fj abert belongs is deter- 
mined. In step 804, it is determined if this pixel is the first 
pixel in the image. If yes, the histogram is simply updated 
in step 805 and steps 802 through 804 are repeated. If the 
pixel is not the first pixel, the bin j for the current pixel is 
compared with the bin b of the previous pixel in step 807. 
If b does not equal j, these two pixels he on a contour of a 
region. This is determined in step 808. The contour pixel is 
coded with a color, e.g., white and the segmented image in 
a second frame buffer (35 in FIGS. 5 and 10) is updated in 
step 810. In step 806, it is determined if the image is 
complete. If not complete, steps 802 to 806 are repeated. If 
j equals b in step 808, the current and previous pixels are in 
the same bin. In this case, the current pixel is coded black at 
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step 809, and the segmented image in the second frame 
buffer is updated in step 805. If in step 806 the end of the 
image is determined, the routine returns at step 811 to step 

259 of FIG. 13. Referring again to FIG. 13, at step 259, the 
encoding statistics (e.g., number of regions, number of 5 
contour pixels) are determined and the routine returns at step 

260 to step 300. 

Referring again to FIG. 12, the most important number is 
the estimation of the number of bits N' required to transmit 
the image which is determined in step 350 from the encoding 10 
statistics. N 1 is compared with the target value of N in step 
400. If N* is greater than N by a value exceeding a tolerance 
6, the segmentation routine must be restarted in step 250 
with a value of the number G of bins in the histogram 
reduced by one in step 450. If N' is close to N, the encoding 15 
may proceed with the routine encoding B in step 500. 

Referring to FIG. 14, routine encoding B steps 501 to 507 
are the steps in which the segmented image with its contour 
pixels, regions and representative values for the pixels are 
output to a memory, to a network, etc. In step 502, the 20 
contour pixels are coded. It is wasteful of bits to output each 
individual pixel. It is more economical to specify only the 
direction of how to get from one contour pixel to the next 
contour pixel. Further, it is wasteful to specify the bound- 
aries between two regions twice, one for each region. 25 
Conventional contour pixel coding schemes provide optimal 
compression of the contour pixel data. In step 503, a header 
may be added, e.g., to specify an address on the Internet. In 
step 504, the region data is coded. The region data is the set 
of representative data which determines the color of each 30 
pixel in a specific region. This data may be output as Lfniben 
data or, alternatively, the LffUben values in the segmented 
image may be output as color components, e.g., RGB or 
CMY values by conversion. The set of representative values 
may be a single value, e.g., the average for the region or the 35 
most common color in the region. Alternatively, a polyno- 
mial may be fitted to the pixel data in each region, and the 
coefficients of the polynomial are then output. DCT and FF 
transforms may be used in a similar way. In step 505, a 
header may be added, and in step 506, the coded region data 40 
is output with the coded contour pixel data as a bit stream to 
a memory or to a communication channel. The routine is 
now complete and returns at step 507 to step 550 of FIG. 12 
to determine if a new image is to be processed, e.g., if the 
images are video frames. If there is no image, the routine 45 
ends in step 600. 

On the receiving end of the coded data, the image must be 
reconstructed by decoding, the flow diagram of which is 
shown in FIG. 17. In step 901, the decoding routine is 
started. In step 902, the data is retrieved, e.g., from a 50 
communication channel such as the Internet. In step 903, 
superfluous data, such as headers is stripped away. In step 
904 the coded contour pixel data is decoded using the 
reverse routine for coding to determine the contour pixels of 
the image to be stored in an output frame buffer. In step 905, 55 
the region data is decoded. This involves decoded the set of 
representative values into color components of each pixel. 
Where the region data has been transmitted as L m i b€rt , 
values, each L Hilbtrt value must be converted into color 
components by a 1-D to 3-D conversion in color space, i.e., 60 
the reverse of the coding routine. The final result is an image 
in the output buffer which is segmented into regions and the 
region pixels have been assigned values from the represen- 
tative set of values for that region. 

The present invention may be particularly useful for the 65 
compression of synthetic or artificial images, e.g., maps with 
color coded height contours, roads, marine charts, etc. or 



cartoon videos. The invention may also find use in video 
conferencing as high compression can be obtained without 
motion compensation, pattern/person recognition as well as 
browsing on the Internet. 

The segmentation and compression routine in accordance 
with the present invention may be used to provide video 
object planes in a multimedia environment. In particular, the 
present invention may find application in MPEG4 environ- 
ments (see drafts of ISO/IEC JTC1/SC29/WG11, publica- 
tions N0998, July 1995; N1028, November 1995, to N1184, 
January, 1996, and N1185, March 1996). 

While the above detailed description has shown, 
described, and pointed out the fundamental novel features of 
the invention as applied to various embodiments, it will be 
understood that various omissions and substitutions and 
changes in the form and details of the system illustrated may 
be made by those skilled in the art without departing from 
the spirit of the invention. 
What is claimed is: 

1. A method of compressing digital data comprising the 
steps of: 

step a) inputting a first matrix of elements, a value of each 
element of the first matrix being represented by a set of 
a plurality of "S" intensity values corresponding to "S" 
quantized levels of a characteristic of each element, 
each set of intensity values associated with an element 
in the first matrix being representable by a coordinate 
of a point in an "S" dimensional space, the "S" dimen- 
sional space having axes representing the intensity of 
the "S" quantized levels of the characteristic as vari- 
ables; 

step b) determining an occurrence matrix (O l1 , . . . 
Qfjf . . . O iW ) on an element-by-element basis for the 
elements in the first matrix, the position of an element 
in said occurrence matrix (O zl , . . . . . . O iW ) being 

determined by the distance from the origin in the "S" 
dimensional space to the coordinate of the point in the 
"S" dimensional space represented by the set of inten- 
sity values for that element as determined by a step- 
wise space filling curve passing uniquely through each 
coordinate of the S-dimensional space; and a value of 
an element (OL^) of the occurrence matrix (0 Llf 



®ln • • • Qlw) being the number of times that a 
coordinate in "S" space represented by the set of 
intensity values of an element of the first matrix has a 
particular distance from the origin of the "S w space 
as determined by the step-wise space filling curve; and 
storing the occurrence matrix (0 L1 , . . . . . . O iW ), 

step c) replacing each element of the first matrix by the 
distance from the origin in the s-dimensional space to 
the coordinate in the S-dimensional space representing 
the set of intensity values associated with that element 
to form a second matrix, and storing the second matrix, 

step d) segmenting the occurrence matrix (O ti , 



O, 



O iTV ) into a variable finite number "F* of 



occurrence segments, and 
step e) determining contour elements of first regions in 
said second matrix, the contour elements of said first 
regions being determined by segmenting the second 
matrix into said first regions using the occurrence 
segments of the occurrence matrix (O l1 , . . . . . - 
O aw ) the distance values in the S-dimensional space of 
each element of the second matrix in one first region of 
the second matrix falling into one of the "P" occurrence 
segments. 
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2. A method of decompressing digital data, comprising the 
steps of: 

step a) receiving digital data representing the contour 
elements of regions in a first matrix and first charac- 
teristic data relating to a characteristic of elements in 
each region of said first matrix each first characteristic 
data associated with an element in the first matrix 
representing the distance from the origin in an "S" 
dimensional space to a coordinate of a point in the "S" 
dimensional space represented by a set of intensity 
values for that element as determined by a step-wise 
space filling curve passing uniquely through each coor- 
dinate of the S-dimensional space and a value of an 
element (OL^) of an occurrence matrix (O l1 , . . . 
®ln • * • °z,w) being the number of times that a 
coordinate in the "S" dimensional space represented by 
the set of intensity values of an element of the first 
matrix has a particular distance from the origin of 
the "S" dimensional space as determined by the step- 
wise space filling curve; 

step b) decoding said digital data to form said first matrix, 
and 

step c) replacing the first characteristic data of each 
element in each region of the first matrix by a set of a 
plurality of "S" intensity values corresponding to "S" 
quantized levels of a second characteristic of each 
element to form a second matrix, each set of intensity 
values associated with an element in the second matrix 
being represented by a coordinate of a point in an "S" 
dimensional space, the "S" dimensional space having 
axes representing the intensity of the "S" quantized 
levels of the second characteristic as variables, the 
coordinates of the point representing the plurality of 
"S" intensity values corresponding to the "S" quantized 
levels of the second characteristic. 

3. The method of claim 1, further comprising the step of: 

f) storing the contour elements as a third matrix, said 
contour elements defining second regions in said third 
matrix corresponding to first regions in said second 
matrix; and 

g) coding the elements of one second region of the third 
matrix with a representative set of values for the 
elements of said one second region, each representative 
value being a distance from the origin in the 
S-dimensional space to a coordinate in the 
S-dimensional space, the set of representative values 
being calculated from the distances from the origin in 
the S-dimensional space to the coordinates in the 
S-dimensional space associated with the elements in 
the first region of said second matrix corresponding to 
the one second region. 

4. The method of claim 1, wherein the step-wise space 
filling curve is one of deterministic or statistical self- 
similarity. 

5. The method of claim 1, wherein the step-wise space 
filling curve is a fractal. 

6. The method of claim 5, wherein the fractal is a Hilbert 
fractal. 

7. The method of claim 3, further comprising the steps of: 
comparing segmentation statistics with a value relating to 

the compression of the first matrix; and 
repeating steps (d) to (g) with a different finite number "P" 
of occurrence segments depending upon the results of 
the comparison. 

8. An apparatus for compressing digital data, comprising: 
input means for inputting digital data in the form of a first 

matrix of elements, a value of each element of the first 
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matrix being represented by a set of a plurality of "S M 
intensity values corresponding to "S" quantized levels 
of a characteristic of said data, each set of intensity 
values associated with an element in the first matrix 
being representable by a coordinate of a point in an "S" 
dimensional space, the "S M dimensional space having 
axes representing the intensity of the "S" quantized 
levels of the characteristic as variables, 
a converter for determining an occurrence matrix 
(0 L1 , . . . . . . O w ) on an element-by-element basis 

for the elements in the first matrix, the position of an 
element in said occurrence matrix (0 L1 , . . . . . . 

Ozw) being determined by the distance from the origin 
in the "S" dimensional space to the coordinate of the 
point in the "S" dimensional space represented by the 
set of intensity values for that element as determined by 
a step-wise space filling curve passing uniquely 
through each coordinate of the S-dimensional space, 
and a value of an element (OL v ) of the occurrence 
matrix (0 Ll9 . . . . . . 0 LW ) being the number of 

times that the coordinate in "S" space represented by 
the set of intensity values of elements of the first matrix 
has a particular distance from the origin of the "S" 
space as determined by the step-wise space filling 
curve: and storing the occurrence matrix (O l1 , . . . 
. . . 0 LW \ 

a first frame buffer for storing the occurrence matrix 



(0 L 



. .0, 



o LW ) , 



segmentation means for segmenting the occurrence 



matrix (p L1 , . . . O a 



O iW ) into a variable finite 



number "P" of occurrence segments, 
a second frame buffer for storing a second matrix, the 
second matrix being formed by replacing each element 
of the first matrix by the distance from the origin in the 
S-dimensional space to the coordinate in the 
S-dimensional space representing the set of intensity 
values associated with that element; 
a segmentation unit for segmenting the second matrix into 
first regions, the distance values in the S-dimensional 
space of each element in one first region of the second 
matrix falling into one of the "P" occurrence segments, 
and for determining contour elements of said first 
regions in said second matrix; and 
a third frame buffer for storing the contour elements of 
said first regions as second regions in a third matrix. 

9. The apparatus of claim 8, further comprising: 
means for coding the elements of one second region of the 

third matrix with a representative set of values for the 
elements of that second region, each representative 
value being a distance from the origin in the 
S-dimensional space to a coordinate in the 
S-dimensional space, the set of representative values 
being calculated from the distances from the origin in 
the S-dimensional space to the coordinates in the 
S-dimensional space associated with the elements in 
the first region of the second matrix corresponding to 
the said one second region in said third matrix. 

10. An apparatus for decompressing data, comprising: 
means for inputting digital data representing contour 

elements of regions in a first matrix and first charac- 
teristic data relating to a characteristic of elements in 
each region of said first matrix, each first characteristic 
data associated with an element in the first matrix 
representing the distance from the origin in an "S" 
dimensional space to a coordinate of a point in the "S" 
dimensional space represented by a set of intensity 
values for that element as determined by a step-wise 
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space filling curve passing uniquely through each coor- 
dinate of the S-dimensional space and a value of an 
element (OL^) of an occurrence matrix (0 L1 , . . . 
On* • • • °zw) being the number of times that a 
coordinate in the "S" dimensional space represented b 
the set of intensity values of an element of the first 
matrix has a particular distance L# from the origin of 
the "S" dimensional space as determined by the step- 
wise space filling curve; 
means for decoding said digital data to form said first 
matrix; 

a converter for converting said first matrix into a second 
matrix, said second matrix being formed by replacing 
the first characteristic data of each element in each 
region of the first matrix by a set of a plurality of "S" 
intensity values corresponding to "S" quantized levels 
of a second characteristic of each element, the "S" 
dimensional space having axes representing the inten- 
sity of the "S" quantized levels of the second charac- 
terstic as variables, the coordinates of the point repre- 
senting the plurality of "S" intensity values 
corresponding to the "S" quantized levels of the second 
characteristic; and 

a frame buffer for storing said second matrix. 

11. The apparatus of claim 8, wherein the step- wise space 
filling curve is one of deterministic or statistical self- 
similarity. 

12. The apparatus of claim 8, wherein the step-wise space 
filling curve is a fractal. 

13. The apparatus of claim 12, wherein the fractal is a 
Hilbert fractal. 

14. The method of claims 1 or 2, wherein the digital data 
comprises a colored image. 

15. The method of claim 14, wherein the colored image 
comprises a map, chart or plan. 
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16. A method of compressing data comprising the Steps 
of: 

providing a colored input image; 
segmenting the image into a segmented image comprising 
5 a plurality of regions defined by contours, wherein the 
segmenting step comprises the steps of: 
scanning the color space of the image based on the 
representation of each color component being dis- 
tributed along an image line and based on a space- 
10 filling curve to form an occurrence matrix, the posi- 

tion of an element in said occurrence matrix being 
determined by the distance from the origin to a color 
coordinate in the color space present in the image, 
and a value of each element of the occurrence matrix 
being the number of times that the color coordinates 
15 are present in the image having a particular distance 

from the origin as determined by the space filling 
curve; and 
storing the occurrence matrix; 
segmenting the image based on the occurrence matrix; 
20 and 

coding said regions and said contours of the segmented 
image, each region being coded depending upon the 
image data in the same region of the colored input 
image. 

2S 17. The method of claim 2, wherein the step-wise space 
filling curve is one of deterministic or statistical self- 
similarity. 

18. The method of claim 2, wherein the step-wise space 
filling curve is a fractal. 
30 19. The method of claim 18, wherein the fractal is a 
Hilbert fractal. 

20. The apparatus of claims 8, 9 or 10, wherein the digital 
data comprises a colored image. 

21. The apparatus of claim 20, wherein the colored image 
35 comprises a map, chart or plan. 

***** 
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